Skip to content

Fix/fw06 restore corpus boundary - #6

Merged
hongdy22 merged 2 commits into
mainfrom
fix/fw06-restore-corpus-boundary
Sep 3, 2026
Merged

hongdy22 merged 2 commits into
mainfrom
fix/fw06-restore-corpus-boundary

Conversation

@hongdy22

@hongdy22 hongdy22 commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

…inst

The stage 2 image refuses to build:

    FATAL: the suite and its recording disagree:
      the recording answers corpus 2c2e7339..., this corpus is 67dfd21e...

The rename before the public release swept the fixed multipart boundary in
corpus.py along with everything else that carried the old name.  It should not
have moved: the boundary is an opaque request input, not a name.  It lands in
the Content-Type header and again in the body, so it feeds fingerprint(), and
data/golden-statea.json was captured before the rename.  Restoring it returns
fingerprint() to
2c2e73398e2c5d2b604ee8bf06ebc7501f0b33a2138936f8023db70cc80e925d, the value the
recording carries.

It went unnoticed because the boundary appears nowhere in the recording itself
-- the upload responses never echo it -- so the rename had nothing to rewrite
there and changed only the input to the hash.

The drift is that token and nothing else.  The 87 case ids already agreed, the
four profiles and their flag lists are untouched, and the fixture aggregate the
recording was captured against still matches the committed docroot.  The
recording itself is not edited here.  Nothing about what this suite measures
changes.

Restored rather than re-captured, for the reason 85cc9ef gives for fw05: a
fresh capture would be a different artifact from the one the published results
were graded against, and the reward would be a nicer spelling for a token that
names nothing.

The audit's provenance gate moves with it.  _ORACLE_TOKENS is there to catch a
submission that recognises the boundary it actually receives, so leaving it
spelled SWERefactorBenchBoundary would have left it searching for a string no
request contains.  It needs restoring in its own right rather than as tidiness:
the old spelling was also caught by _HARNESS_TOKENS' "SWERefactor" prefix, and
the restored one is not, so that gate is now the only thing covering it.

Verified by building the image: the self-check passes, 87 records against the
restored fingerprint, and the suite still collects 478 checks.

Reported in #5.
… build

The fingerprint check that caught the fw06 drift lives in each task's stage
Dockerfile, so it only runs when an image is built.  That is how a rename
reached a release with the corpus and the recording disagreeing, and it is why
the drift was reported from outside rather than found here.

The same comparison runs in CI now: four suites, one second, no docker.  It
re-computes each stored fingerprint from the harness that would send the
requests, and for fw06 it also compares the record ids against the case ids.

The second test is the one that keeps this honest.  It scans every behavioural
data directory for a stamped fingerprint and fails if the set of suites that
stamp one has drifted from the set checked here, so a new suite cannot ship a
recording that nothing re-computes.

Verified in both directions: green as committed, and red on fw06 with the
renamed boundary put back.
@hongdy22
hongdy22 merged commit f8ba808 into main Sep 3, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant